[PECO-1803] Databricks sqlalchemy is split into this folder #1

jprakash-db · 2024-08-20T17:54:06Z

Description

databricks_sql_python is split into the core and databricks sqlalchemy part, the databricks sqlalchemy part is moved into this repo for controlling its release events

Tasks Completed

Moved the databricks_sqlalchemy into this repo
Made necessary code changes to the repo so that post split everything works as intended
Changed the pyproject.toml files to handle the dependencies properly
Added Github actions to run the tests using the databricks_sql_connector_core

How to Test

Testing can be done using the pytest library in the same as it is mentioned in the README.md file

jackyhu-db · 2024-09-06T07:28:23Z

README.md

+
+```shell
+pip install databricks-sql-connector[sqlalchemy]


should be databricks-sql-sqlalchemy?

@jackyhu-db Fixed it

jackyhu-db · 2024-09-11T00:23:08Z

README.md

+
+```shell
+pip install databricks-sql-connector[databricks_sqlalchemy]


it should be directly installed pip install databricks_sqlalchemy

jackyhu-db · 2024-09-11T00:24:32Z

.github/workflows/integration.yml

+      - name: Checkout Dependency Repository
+        uses: actions/checkout@v3
+        with:
+          repository: jprakash-db/databricks-sql-python


do not use personal repo and it should be databricks/databricks-sqlalchemy

jackyhu-db · 2024-09-11T00:25:01Z

.github/workflows/integration.yml

+        with:
+          repository: jprakash-db/databricks-sql-python
+          path: databricks_sql_python
+          ref : jprakash-db/PECO-1803


what is this?

This was a temporary fix to check the integration tests are passing or not. Once the main split gets published I need to modify this

jackyhu-db · 2024-09-11T00:25:58Z

.github/workflows/integration.yml

+          python -m pip install --upgrade pip
+          pip3 install poetry
+          python3 -m venv venv
+          ls databricks_sql_python/databricks_sql_connector_core


why ls databricks_sql_python/databricks_sql_connector_core, it does not exist

jackyhu-db · 2024-09-11T00:26:49Z

.github/workflows/integration.yml

+      - name: Build Dependency Package
+        run: |
+          source venv/bin/activate
+          pip3 install databricks_sql_python/databricks_sql_connector_core/dist/*.whl


you should install via pip install requirements.txt which has the package databricks_sql_connector_core as the dependency

But currently the databricks_sql_connector_core is not a published library and that is why all these changes are made just for testing. Once we decide how to publish I will change the integration.yml file

jackyhu-db · 2024-09-11T00:28:19Z

README.md

+
+```shell
+pip install databricks-sql-connector[alembic]


again, we should use pip install databricks_sqlalchemy, do not refer to databricks-sql-connector, which may confuse the user

jackyhu-db · 2024-09-25T06:50:08Z

README.tests.md

what is this file?

jackyhu-db · 2024-09-25T06:51:41Z

pyproject.toml

+pytest-dotenv = "^0.5.2"
+
+[tool.poetry.urls]
+"Homepage" = "https://github.com/databricks/databricks-sql-python"


should it be https://github.com/databricks/databricks-sqlalchemy?

jackyhu-db · 2024-09-25T06:53:01Z

src/databricks_sqlalchemy/_ddl.py

again, you change the package name, it was databricks.sqlalchemy, please make sure it is backward compatible

jackyhu-db

You change the package name, it was databricks.sqlalchemy, please make sure it is backward compatible. Use databricks_sqlalchemy will break existing clients

…ent globally

jackyhu-db

I assume you have not changed any source file under src/databricks/sqlalchemy

…)" This reverts commit 76a3c28.

databricks-sqlalchemy plugin split

53e63ab

jprakash-db requested a review from madhav-db August 20, 2024 17:54

jprakash-db self-assigned this Aug 20, 2024

jprakash-db added 4 commits August 22, 2024 22:30

Refractored code and added pytest.ini for sqla_testing issue

91d6cd2

Changed imports from databricks.sqlalchemy to databricks_sqlalchemy

4bc939b

Added the integration.yml workflow

dc0a25b

Changed screts name

433c172

jprakash-db removed the request for review from madhav-db August 26, 2024 04:17

jprakash-db marked this pull request as ready for review August 26, 2024 04:18

Added workflow to run on commit changes

68fa020

jprakash-db had a problem deploying to azure-prod August 26, 2024 08:32 — with GitHub Actions Failure

jprakash-db had a problem deploying to azure-prod August 26, 2024 08:37 — with GitHub Actions Failure

jprakash-db had a problem deploying to azure-prod August 26, 2024 08:39 — with GitHub Actions Failure

jprakash-db had a problem deploying to azure-prod August 26, 2024 08:46 — with GitHub Actions Failure

jprakash-db had a problem deploying to azure-prod August 26, 2024 08:49 — with GitHub Actions Failure

jprakash-db had a problem deploying to azure-prod August 26, 2024 08:51 — with GitHub Actions Failure

jprakash-db had a problem deploying to azure-prod August 26, 2024 08:54 — with GitHub Actions Failure

jprakash-db had a problem deploying to azure-prod August 26, 2024 08:56 — with GitHub Actions Failure

jprakash-db had a problem deploying to azure-prod August 26, 2024 08:59 — with GitHub Actions Failure

jprakash-db had a problem deploying to azure-prod August 26, 2024 09:02 — with GitHub Actions Failure

jprakash-db had a problem deploying to azure-prod August 26, 2024 09:05 — with GitHub Actions Failure

jprakash-db had a problem deploying to azure-prod August 26, 2024 09:07 — with GitHub Actions Failure

jprakash-db had a problem deploying to azure-prod August 26, 2024 17:26 — with GitHub Actions Failure

jprakash-db had a problem deploying to azure-prod August 26, 2024 17:29 — with GitHub Actions Failure

jprakash-db had a problem deploying to azure-prod August 27, 2024 06:12 — with GitHub Actions Failure

jprakash-db had a problem deploying to azure-prod August 27, 2024 06:46 — with GitHub Actions Failure

jprakash-db had a problem deploying to azure-prod August 27, 2024 06:54 — with GitHub Actions Failure

Fixed issue of failing tests in github actions

296c5a3

jprakash-db force-pushed the PECO-1803 branch from 7b9dd81 to 296c5a3 Compare August 27, 2024 07:12

jprakash-db had a problem deploying to azure-prod August 27, 2024 07:12 — with GitHub Actions Failure

jprakash-db temporarily deployed to azure-prod September 6, 2024 06:42 — with GitHub Actions Inactive

jackyhu-db reviewed Sep 6, 2024

View reviewed changes

README.md edits

786c0be

jprakash-db temporarily deployed to azure-prod September 7, 2024 04:58 — with GitHub Actions Inactive

jackyhu-db reviewed Sep 11, 2024

View reviewed changes

Renamed the imports to databricks

a7bbc5a

jprakash-db temporarily deployed to azure-prod September 13, 2024 04:01 — with GitHub Actions Inactive

jprakash-db mentioned this pull request Sep 18, 2024

[PECO-1803] Splitting the PySql connector into the core and the non core part databricks/databricks-sql-python#443

Closed

5 tasks

Minor refractors

073781e

jprakash-db had a problem deploying to azure-prod September 24, 2024 08:01 — with GitHub Actions Failure

Removed the Integration.yml file

52c0a29

jackyhu-db reviewed Sep 25, 2024

View reviewed changes

README.tests.md Outdated

Copy link

Contributor

jackyhu-db Sep 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is this file?

jackyhu-db reviewed Sep 25, 2024

View reviewed changes

jackyhu-db requested changes Sep 25, 2024

View reviewed changes

jprakash-db added 3 commits September 25, 2024 22:33

Changed the folder structure to have databricks as the common namespace

ef0ee61

Minor Fix

a317b39

Removed the README.md from within the sqlalchemy folder as it is pres…

5ac49fe

…ent globally

jackyhu-db approved these changes Sep 26, 2024

View reviewed changes

jackyhu-db reviewed Sep 26, 2024

View reviewed changes

jprakash-db mentioned this pull request Sep 26, 2024

PySQL Connector split into connector and sqlalchemy databricks/databricks-sql-python#444

Open

7 tasks

Fixed LICENSE and other issues

144907c

jprakash-db merged commit 76a3c28 into main Oct 22, 2024

jprakash-db added a commit that referenced this pull request Oct 22, 2024

Revert "[PECO-1803] Databricks sqlalchemy is split into this folder (#1…

2a6f6fb

…)" This reverts commit 76a3c28.

jprakash-db mentioned this pull request Oct 22, 2024

Revert "[PECO-1803] Databricks sqlalchemy is split into this folder (… #2

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[PECO-1803] Databricks sqlalchemy is split into this folder #1

[PECO-1803] Databricks sqlalchemy is split into this folder #1

jprakash-db commented Aug 20, 2024 •

edited

Loading

jackyhu-db Sep 6, 2024

jprakash-db Sep 7, 2024

jackyhu-db Sep 11, 2024

jackyhu-db Sep 11, 2024

jackyhu-db Sep 11, 2024

jprakash-db Sep 24, 2024

jackyhu-db Sep 11, 2024

jackyhu-db Sep 11, 2024 •

edited

Loading

jprakash-db Sep 24, 2024

jackyhu-db Sep 11, 2024

jprakash-db Sep 24, 2024

jackyhu-db Sep 25, 2024

jackyhu-db Sep 25, 2024

jackyhu-db Sep 25, 2024 •

edited

Loading

jackyhu-db left a comment

jackyhu-db left a comment


		```shell
		pip install databricks-sql-connector[databricks_sqlalchemy]

[PECO-1803] Databricks sqlalchemy is split into this folder #1

[PECO-1803] Databricks sqlalchemy is split into this folder #1

Conversation

jprakash-db commented Aug 20, 2024 • edited Loading

Related Links

Description

Tasks Completed

How to Test

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jackyhu-db Sep 11, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jackyhu-db Sep 25, 2024 • edited Loading

Choose a reason for hiding this comment

jackyhu-db left a comment

Choose a reason for hiding this comment

jackyhu-db left a comment

Choose a reason for hiding this comment

jprakash-db commented Aug 20, 2024 •

edited

Loading

jackyhu-db Sep 11, 2024 •

edited

Loading

jackyhu-db Sep 25, 2024 •

edited

Loading